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ABSTRACT 



This paper reports outcomes from a development and 
evaluation effort focused on a specific World Wide Web-based learning 
environment (WBLE) for school library media specialists (SLMS) . Background 
information is provided explaining how the decision to create a WBLE for the 
SLMS course was derived. The three main groups contributing to the WBLE 
process are described, which include course designers/developers, course 
facilitators, and course participants. Issues associated with the creation 
and implementation of the WBLE are discussed in the context of two phases. 
Design/development issues during the first phase (i.e. the design and 
development phase) included page vs. site design and cross-platform 
performance; logistical/management issues included ideals vs. reality, site 
access, and on-site assistance; technological challenges included modems, 
browsers, and network stability. During the second phase (i.e., 
implementation and on-going evaluation) , design/development issues included 
network stability, Web browsers, content delivery vs. data gathering; 
logistical/management issues included mixing face-to-face and distance 
interactions, the importance of flexibility and feedback, and keeping the 
learners motivated. Perspectives of designers/developers, instructors, and 
learners are offered for clarification and understanding of the potential 
impact and promise of WBLE s . Includes sample course Web pages). (AEF) 
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Abstract 

WBI offers a new and exciting avenue for meeting the needs of students, particularly the needs of 
those students who would not otherwise be able to engage in a formal educational process. WBI 
enables continued involvement in the learning process in an anytime, anyplace orientation. While 
the promises of Web-based instruction are great, delivery on these promises has yet to be realized. 
One reason for the delay may be the lack of shared experiences for how to create and develop these 
environments. The purpose of this paper is to report outcomes from a development and evaluation 
effort focused on a specific Web-based learning environment (WBLE) for school library media 
specialists (SLMS). 
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Web-Based Delivery Of Instruction: Prospects And Challenges 

The number of Web sites currently accessible to the everyday Web-surfer is staggering. 
Since the creation of the Web in 1992 by Tim Berners-Lee of CERN (a collective of European 
high-energy physics researchers), millions of Web pages have been added to the milieu and the 
number continues to increase at an exponential rate. Latest figures estimate that there are over 300- 
million pages on the Web (Haverkamp & Gauch, 1998; Edwards, 1998). The number of Web 
sites devoted to educational purposes is also increasing. As instructional Web sites continue to 
grow, they bring with them a message of enhanced educational environments and student 
satisfaction. The literature is filled with stories told by instructors and learners engaged in Web- 
based instruction (WBI) touting its merit and encouraging its use (Bazillion & Braun, 1998). 

WBI offers a new and exciting avenue for meeting the needs of students, particularly the 
needs of those students who would not otherwise be able to engage in a formal educational 
process. WBI also enables continued involvement in the learning process in an anytime, anyplace 
orientation. This can be particularly appealing to students who are dispersed across a broad 
geographic area and/or want to continue with their formal education, but need the convenience 
WBI can afford. Indeed, WBI does hold a great deal of potential for the delivery of instruction 
(Owston, 1997). 

While the promises of Web-based instruction are great, delivery on these promises has yet 
to be realized. One reason for the delay may be the lack of shared experiences for how to create 
and develop these environments. To the credit of those working in this area, not much time has 
elapsed since the explosion of WBI and the need for this shared knowledge base. While some 
work has been published [see Khan (1997) for a compilation of chapters related to WBI] the need 
for more shared experiences and guidelines for creating and developing WBI does exist, and 
continues to increase daily. 

In an effort to contribute to the knowledge based for creating WBI, we will report 
outcomes from a development and evaluation effort focused on a specific Web-based learning 
environment (WBLE) for school library media specialists (SLMS). Background information will 
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be provided explaining how the decision to create a WBLE for the SLMS course was derived. The 
three main groups contributing to the WBLE process will be described, including course designers/ 
developers, course facilitators, and course participants. Issues associated with the design and 
development of the WBLE, including the logistical, management and technological challenges, will 
be discussed. Perspectives of designers/developers, instructors and learners will be offered for 
clarification and understanding of the potential impact and promise WBLEs pose. 

WBI: OPPORTUNITIES AND POSSIBILITIES 

During the spring of 1997, the opportunity to reach a geographically dispersed population 
of graduate students in a Western state via distance technologies was presented to an Educational 
Technology program. In this Educational Technology program, one of the most geographically 
dispersed audiences is the group of students seeking a degree in Educational Media. These 
students, adults (25 - 50) working full-time, are located throughout the state (a geographic region 
covering more than 100,000 square miles) and across several school districts. As one of the only 
Educational Media programs in the state offering a degree preparing students for certification in 
educational media, the demands placed upon the program are high and continue to increase. These 
factors made the decision of which educational technology program to target for distance delivery 
an easy one for the program faculty. 

Several courses are offered as a part of the Educational Media program. The course which 
is the topic of this paper, ET 533: Reference and Information Management, is one of four core 
courses in the program. ET 533 was selected as the course to begin exploration of offering the 
Educational Media program at a distance for a variety of reasons: 

♦ the course involves a great deal of interaction, something which can be 
facilitated at a distance; 

♦ many students who were registered to take the course in the Summer of 
1997 lived in remote locations from campus; and 
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♦ the lead author of the paper was interested in continuing research in 

the area of WBI and to assist learners obtain information in an 

anytime, anyplace orientation. 

ET 533: Reference and Information Management 

The primary goal of the ET 533 course is to assist learners in developing an understanding 
of the reference process and the activities involved in information management. In developing this 
understanding, emphasis in the course is placed on the exploration of the various types of 
materials, databases, and reference services that support K-12 curricula. The learner engages in 
several activities, including: 

• Participation in discussions related to the "Topic of the Week" (e.g., Reference 
on the InfoBaun); 

• Completion of three simulation activities (e.g., Electronic Information); 

• Reference tools comparison and contrast paper; 

• Electronic networks comparison and contrast paper; 

• Reference collection plan; 

• Lesson plan; and 

• Personal philosophy paper. 

Each of the activities are designed to assist the learners in developing an understanding of the 
nature of the reference process, information sources, and evaluation of reference services. 

ET 533: Participants 

Three groups participated in the delivery of ET 533 at a distance: course 
designers/developers, course facilitators, and the learners. 

Course Designers/Developers 

The two authors from the University of Northern Colorado (UNC) served as the lead 
designers and developers of the Web site. Together, Dr. Hill and Mr. Murry brought a combined 
8 years of experience to the effort of creating ET 533 for Web-based delivery. 

In addition to Dr. Hill’s and Mr. Murry experience, the experience of other professionals 
was also utilized during the design and development of the course Web site. Three groups of 
experts were used during the creation of ET 533: subject-matter experts in the school library media 
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area (n=4), instructional design experts (n=6), and distance education experts (n=2). Expert 
review took place during three phases of the design and development process: design, production 
and implementation. The feedback received from the evaluators was used to inform all design and 
development decisions by the lead designers. 

Course Facilitators 

All three authors served as course facilitators for ET 533. Dr. Hill and Dr. Rezabek 
brought a combined 25 years of experience in teaching courses in the library media/instructional 
design area. Mr. Murry brought his expertise in the use of the course WBLE to ET 533 and was 
valuable in assisting the learners in becoming comfortable in working in the on-line environment. 
Learners 

The learners (n=12) enrolled in the ET 533 course in the summer of 1997 were all 
participants in the Educational Media program, seeking State certification as media specialists. The 
majority of the learners enrolled in ET 533 were working in school library media centers at the K- 
12 level. In terms of experience with the Web, the learners varied considerably. Some possessed 
considerable experience; others very little. None of the learners had engaged in formal a 
educational experience via the Web. A final characteristic the learners shared was a willingness to 
try new things and a desire to make the learning process easier for them from a logistic orientation. 
These two factors are what really made the offering of ET 533 at a distance possible. 

ET 533: Delivery Decisions and Development Support 

In seeking to meet the goals of the course and to best meet the needs of the learners, a 
variety of delivery options were discussed as a result of the opportunity to deliver the Educational 
Media program, starting with the ET 533 course, at a distance (e.g., compressed video, videotaped 
sessions, windshield and flying faculty models). The distance delivery mode selected was a Web- 
based implementation for the course (see Figure 1-3 for examples of the pages from the Web site). 
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The decision of delivery method was made based on several factors: a preliminary analysis 
of the audience; a review of the goals for the class and how they might best be met; and an 
examination of the technology available for design and delivery of the instruction. While some 
delivery options were rapidly discounted, many of the options were feasible and desirable on a 
variety of levels. Ultimately, the most compelling factors driving the delivery method decision 
were accessibility and convenience for the student and the course facilitators. 

To enable the creation of a WBLE for the course, the lead author of the paper obtained a 
grant to fund a research assistant, Mr. Murry, to assist with the design and development of the 
course. Together, the two authors from UNC created initial plans for the Web-based 
implementation of the course, following through with actual development. The second author also 
assisted with refinement of the design, and was instrumental in co-facilitation of the course during 
its implementation. 

ET 533: The Creation of a WBLE 

The design, development, and implementation of ET 533 took place during a three-month 
period. Activities can be divided into two stages: prior to implementation of the course and during 
the course. In building a context for the reader, challenges and issues related to the creation and 
implementation of the WBLE will be discussed within these phases. 

Phase One: Design and Development 

After the grant was funded, the two authors from UNC worked on creating storyboards 
and creating initial ideas for the ET 533 WBLE. A constructivist approach to the creation of 
distance education was used for designing the ET 533 Web site (Jonassen et al., 1995). As stated 
by Jonassen et al. (1995): "Constructivist environments engage learners knowledge construction 
through collaborative activities that embed learning in meaningful context and through reflection on 
what has been learned through conversation with other learners" (p. 13). Several activities in the 
course support a constructivist approach to distance education as discussed by Jonassen et al. 
These include: deriving solutions to simulations/cases related to the reference process embedded in 
the K-12 environment; the completion of two reflection papers related to the learner's personal 
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articulation of the reference process; collaboration and conversation in small and large groups in 
order to reach consensus in meaning making. 

A rapid prototyping approach was used for the development of the WBLE. The rapid 
prototyping approach to design and development involves continuous evaluation and revision to an 
instructional product/environment as it is being created (Tripp & Bichelmeyer, 1990). For the pre- 
implementation phase of the course, various experts, following a connoisseur type of evaluation as 
described by Flagg (1990) conducted evaluation through multiple reviews. The authors spent an 
intensive month and a half developing the course storyboards and ideas into Web pages for the 
WBLE, revising and improving the site based on expert evaluation data. 

Several issues and challenges were encountered during the creation of ET 533. These are 
briefly discussed in the following sections: design/development issues, logistical/management 
issues, and technological challenges. 

Design/Development Issues 

Page vs. Site Design. How to best structure the creation of the ET 533 WBLE was a 
major concern to the designers/developers. The question of structure related to designing the site 
from a page by page perspective, or designing the ET 533 WBLE on a more global level — site 
design. The decision was made to follow a more global orientation in creating the site. Thus, 
when you view the individual pages, there is an inherent "look" and "feel" to the entire site. 

Cross-Platform Performance. Another design/development issue related to the ability to 
access the site regardless of the platform. In our context, there were two primary systems used to 
access the ET 533 WBLE: Macintosh and Windows. Several issues arose as we developed the 
site, including: 

♦ a need to view the Web site on both platforms as well as different size monitors to see 
what the user will be viewing. Functionality, sytle and cosmetics cannot be assumed to 
be appropriate, and it is never safe to assume WYSIWYG! 

♦ a need to "code" for downward compatibility. This issue was particularly relevant in the 
creation of the navigational bar on the bottom of the Web site (see Figure 1). The pop 
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up windows for navigation could be seen on the Mac while there were problems in the 
Windows environment. This created a need to adjust the pull-up menus to 
accommodate both platforms. 

Logistical/Management Issues 

Ideal Vision Meets Reality. As mentioned earlier, we were working under a very tight 
deadline to get the site up and running before classes started. As a result, functionality became the 
order of the day. Several items quickly became "nice to have" as we worked to get the site up and 
running in a month and a half. 

Site Access. One logistical/management issue related to where students would be accessing 
the site. We assumed that students would be accessing from ‘home’ — then found that since the 
class was advertised as on-campus, we had to offer it on campus. As related later in this paper, we 
again had to shift in midstream to offer off-campus access options that created a brand new set of 
problems. 

On-site Assistance. An associated issue is that we did have on-site help planned (Brandon 
and other lab helpers) from the on-set of the course. This forethought may have deterred the 
growth of other logistical/management issues. Further developments discussed later emphasized 
the importance of on-site technical assistance. 

Technological Challenges 

Modems and Browsers. The hardware and software used to access the ET 533 were also a 
concern during pre-implementation of the course. We beta tested the site using a variety of 
browsers (e.g., Netscape Navigator, Netscape Communicator, Internet Explorer) and versions, as 
well as a variety of different speed modems. We quickly learned that the optimal browser for ease 
in access was Netscape Navigator 3.0 (or higher) and a 14.4 modem. 

Network Stability. As mentioned earlier, the network at the University were the ET 533 
Web site was being served was far less than reliable. It was not unusual for the network to "go 
down," creating access issues for the designers/developers as they worked on the creation of the 
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site. Lack of access also created challenges for the evaluators of the site. To ensure that data was 
received, we also asked evaluators to send in their evaluations via surface mail. 

Phase Two: Implementation and On-going Evaluation 

Once the course was underway, refinement and enhancement of the course remained as on- 
going activities. Design decisions were reconsidered; development of the course continued 
throughout course implementation. 

Several issues and challenges were encountered during the implementation of ET 533. 
These are briefly discussed in the following sections: design/development issues and 
logistical/management issues. 

Design/Development Issues 

Network Stability. One of the biggest challenges encountered during the implementation of 
the course related to the University network. The University servers were constantly going down. 
This affected the students' access to the Web site, as well as their ability to send e-mail and 
participate in the listservs. 

Web Browsers. Another issue related to the Web browser students were using to access 
the site. While the students were informed that Netscape Navigator 3.0 was the preferred (and 
most reliable) browser, it was not always possible for students to access the site via Navigator. As 
a result, two sites were developed: dynamic and basic. The dynamic version of the ET 533 
WBLE relied upon JavaScript code to function. The basic version of the WBLE used basic HTML 
code. The offering of the two versions enabled easier access for all of the learners. 

Content Delivery vs. Data Gathering. Yet another issue related to the student's 
understanding of the experimental nature of the ET 533 WBLE. While the course was being 
offered for the delivery of content as a part of the Educational Media program, it was also being 
developed as a research project. Since these were students majoring in Educational 
Technology/Educational Media, we talked about the practice of formative evaluation and revision. 
Though we were almost constantly dealing with technical issues that influenced students’ ability to 
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access and use the site, they had a good lesson in some of the real world frustrations of 
instructional design and development. 

Logistical/Management Issues 

Mixing F2F and Distance Interactions. As indicated earlier, the course was initially 
intended to be taught off-site, then was required to be taught on-site. Since one of the motivating 
factors for students to engage in WBLE study is the flexibility of time and place, ‘forcing’ students 
to come to class all at the same time in the lab seemed to defeat the purpose. However, since we 
had so many problems with the system/technology, neither could we really experience a totally 
WBLE setting. So we compromised. 

At first, all of us met at the same time in the classroom lab, got online, and pretended we 
were online at the same time at different locations. Then students who chose to do so accessed the 
WBLE from their home or work systems while students who chose to come to class worked from 
the lab-all during regularly scheduled class time. The final “phase” allowed students to participate 
in learning activities from any site they chose (and could access the ET 533 site) and at a time of 
their choice. This situation was closest to the “real world” model of WBLEs, though about 30% of 
the students still came to the class lab during regular class time. The major reason appeared to be 
technical problems with access to and performance of the WBLE from varied off-campus sites. It 
was easier to come to class than to struggle with the hardware. 

The Importance of Flexibility and Feedback. Students were told by the on-site instructor at 
the beginning of class that prerequisites included flexibility, an open mind, and a sense of humor. 
Their response to numerous technical problems, updates and revisions actually prompted important 
changes and additions to the WBLE. For example, the students complained about the confusing 
nature of having all types of discussions occurring simultaneously in one chat room, and Mr. 

Murry revised the chat room to accommodate ‘threads’ of conversations. 

Keeping the Learners Motivated. Students were good sports about ‘pretending” they were 
accessing the WBLE from different locations at different times. The situation also provided 
numerous opportunities to discuss sociological aspects of WBLE and on-line information access. 
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One motivating factor we kept stressing when students balked at using the WBLE was that much 
information in media centers is e-based these days. Therefore it seemed particularly appropriate to 
learn about online resources using a WBLE. 

Multiple Perspectives: Designers, Instructors, and Learners 

While several challenges and obstacles were encountered during the design of ET 533 and 
when the course was implemented, analysis of the data gathered as a part of the on-going 
evaluation of the course revealed that all audiences were pleased with the outcomes of the course. 
Specifics are discussed in the following sections. 

Evaluation Techniques 

The evaluation of the course was approached from an inductive paradigm; that is, the 
evaluation did not begin with a predictive hypothesis. Rather, a bottom-up approach was taken to 
discern patterns in the data, with themes generated to explain effective and ineffective aspects of the 
learning environment (Flagg, 1990). Self-report techniques were used to gather data from the 
course designers/developers, course facilitators, and course participants. In discussing the results 
of the evaluation, specific reactions and feedback will be presented in three areas: course 
designers/developers, course facilitators, and course participants. 

Course Designers/Developers 

Feedback on the ET 533 WBLE design and development experience was gathered 
retrospectively from the course designers/developers. Overall, the course designers/developers 
found the experience to be quite an adventure. However, it was not without hazards. 

One of the biggest challenges faced in the design/development of the ET 533 WBLE was 
the consistency and style of the site. Considerable time and energy was devoted to the "feel" of the 
site. While it did reap rewards in terms of the students overall reactions (see comments in section 
below), it was a long, hard battle some days. 

Another major challenge related to the dynamic nature of working on the Web. Things are 
not stable in this environment, often bringing changes that create unanticipated havoc. For 
example, a Web page was developed which emulated the download site for Netscape. We did this 
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to assist the students in navigating the Netscape Web site and to help ensure that they downloaded 
the correct version of Navigator. The day before we went online with the ET533 site, Netscape 
made major changes to their download site (address and cosmetics and function). This created a 
need to either update the emulation page on the ET 533 site, or to delete the page from the Web 
server. We chose to drop the emulation of the Navigator pages and provide only the link to the 
pages via Netscape. 

Hardware was (and remains) a major concern — especially in terms of the quality and 
consistency of student access. We learned we certainly can create and implement WBLEs. 
However, at least two fundamental questions remain to be answered: can we do it well, and can we 
do it well enough to be cost effective and worthwhile. 

Course Facilitators 

Like the feedback from the course designers/developers, feedback from the ET 533 WBLE 
course facilitators was gathered retrospectively. Overall, the course facilitators found the 
experience to be rewarding, though challenging. 

One of the challenges mentioned by the course facilitators related to technical support. As 
stated by Dr. Rezabek, "Yes, the technical support is still a major concemV' All of the students in 
the ET 533 course were novices in the use of the Web for formal educational purposes. Many of 
the students had never engaged in a distance education environment of any form. This created a 
need for extensive technical support throughout the course. 

Another challenge mentioned by the facilitators related to the need for flexibility. 

According to Dr. Rezabek, the flexibility had to be present from the learners as well as the 
facilitators: "As a facilitator, I asked [the] students to be flexible, open-minded, risk-taking 
participants who maintained a sense of humor. I had to be the same way\ " Dr. Hill echoed the 
need for flexibility, open-mindedness and risk-taking, but with a very different perspective: 

I was working at the Johnson Space Center when the course was 
implemented. This put me in a unique situation, where as a facilitator I 
was at a distance while the learners were on-site for some face-to-face 
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interactions. I had to be extremely flexible all around — in terms of the 
hardware I was using, in terms of when assignments were submitted, 
and in terms of the times I engaged in interactions with the learners. It 
was a real challenge some days, but incredibly rewarding! 

While there were challenges, there were also substantial rewards. One of the “neatest 
things” that happened was that we talked (face-to-face as well as online) about the social impact of 
online information retrieval (after all, this was a reference class!). That discussion provided a 
remarkable segues to discuss the social and practical aspects of participating in a WBLE. The 
content of the reference course, with its emphasis on media specialists looking forward to new 
learning environments and reference demands in the 21st century, provided a wonderful context to 
“coax” any somewhat reluctant students to participate in the online aspects of the course. 

Another very healthy aspect of this course was that students saw faculty continuing to 
leam. We made mistakes and admitted them! We asked students to lend their expertise and to 
collaborate. We took risks and allowed everybody involved to make mistakes and try again! Even 
though these may not be accepted as crucial aspects of WLBE, they proved remarkably effective in 
building a learning community. 

Team teaching is draining, hard, frustrating, but incredibly growth provoking both for the 
instructors involved and for the students. It was especially interesting since Dr. Hill was 
distanced. She was the lead instructor and primarily responsible for providing feedback on 
students’ projects and performance. Though both instructors did answer student questions, there 
was a definite tendency for students to ask Dr. Rezabek questions first just because she was bodily 
present two days a week — though she was inaccessible at all other times! 

Overall, the facilitators concluded that the experience was extremely challenging, but 
incredibly rewarding. And the questions related to the creation and implementation continues to 
arise. As stated by Dr. Rezabek: "There is so much more to leam! I’m especially curious about 
social interaction and learning online." Dr. Hill stated: "/ continue to be fascinated by the concept 
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of community building in on-line environments. I know it happens — I saw it in ET 533 — I am 
just not sure how or why. I would echo Landra: there is so much more to learn!" 

Learners 

While the information gleaned from the designers/developers and facilitators is important, 
perhaps the most critical voice in the evaluation process is that of the participants or learners. The 
learners participating in the ET 533 course were like most adult learners: more than willing to 
share their opinions and to direct their own learning activities. The learners’ ability and willingness 
to share their opinions was critical in the continual refinement and enhancement to the course 
WBLE. 

Evaluation data was formally gathered from the learners at two stages of the course: mid- 
term and at the conclusion of the course. While the learners were very willing to let the facilitators 
know when things were not running smoothly throughout the course, the formal gathering of 
feedback proved critical to the continual improvement of the course. The data presented below has 
been broken out into major themes identified during the analysis of the learners' feedback at mid- 
term and then again at the conclusion of the course. 

Mid-Term Evaluation 

As a part of the mid-term evaluation of the course, learners were asked to respond to three 
(3) statements related to the course: 

1 . List three “neat things” you have learned so far. 

2. Discuss any frustrations and/or successes you have experienced in working 
with the WBLE. 

3. Discuss “how it’s going” overall. 

The following themes were evident in their responses. 

Question One: List three ‘‘neat things” you have learned so far. 

Theme One: Experiences of Others. The ability to draw upon the experience of peers 
working in the field (i.e., K-12 library media centers) was an important aspect to many of the 
learners in the course. The ability to share experiences appeared to be enhanced by the electronic 
technologies used in ET 533 to facilitate interaction. As stated by one learner in the course: 
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There are some really good and knowledgeable people in this class that 
have a lot of good ideas and experiences to draw from. A few of the 
people have spent some time on the job already, and it’s interesting to 
draw on their stories when we talk about the issues. 

Another learner in the course echoed these sentiments: 

I enjoy using the Internet and sending e-mail. I like the opportunity to 
discuss questions in the weekly chat sessions. It’s interesting to read 
others opinions and thoughts. I feel I am benefiting from talking with 
class members who have experience working in media centers and share 
their problems/solutions.... 

Theme Two: How to use Tools. The ability to use the tools (e.g., e-mail, listservs, Web) 
was also reported by several ET 615 students as a “neat thing” they learned by mid-term. Several 
of the students reported have little to no experience with Internet technologies.. The fact that they 
were able to feel more comfortable with the tools was an important accomplishment. As stated by 
one learner: 

...I’ve learned how to access [the] Internet which was an overwhelming 

/ 

feat individually. [In the past,] I never found the information I wanted 
and it seemed I was pulled in a million directions. I’ve been given a 
solution to that. I’m now less intimidated by the process... 

For another learner, the number of tools they were able to use by the mid-point of the course was 
an important achievement: 

I am learning more about using E-mail every day.... I have also been 
learning how to access and use the WBLE site. These have been new 
concepts for me. I have particularly enjoyed answering the questions 
using the chat areas. 

Theme Three: Reference Techniques. Another “neat thing” learned mentioned by ET 533 
students at mid-term related to the reference process and how they could improve their performance 
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as media specialists assisting students with finding needed information — whether electronic or 
physical. For some students, this meant learning techniques for searching for information on the 
Internet. As one learner stated, "I've learned Boolean techniques to help searches on the Internet— 
what a great thing to know." Another learner stated: "[I learned that] putting quotes (" ") around a 
subject when entering it in to search the Web limits it to 4 (four) hits or so and not 4,000." 

For other students, interaction techniques where enhanced: "I have learned that questioning 
my patrons will help me manage my time when assisting me with my reference searches." Another 
student echoed this thought when they stated: "[I have learned] the importance of a well formed 
reference question." One student related the reference process to personal interaction: 

The personal aspect is the most important when providing reference 
services. If the reference librarian is not approachable, interested and 
gifted with people skills, patrons will struggle with their reference 

issues. 

Another area students mentioned in relation to reference techniques feel more in the area of 
reassurance — reassurance that they were not doing things "wrong" or that they should not expect 
themselves to know everything. Several comments were made by students related to this area: 

♦ I have learned that everyone has as many questions about how we will 
be able to meet all the information demands of the future. 

♦ I haven't been doing everyone incorrectly!!! ;-) Although I may seem 
somewhat insecure as far as being a subject-area expert, my technology 
and communication skills should help carry me through. 

♦ [I learned] there are many more research sources available than I ever 
thought possible or knew about. 

Question Two: Discuss any frustrations and/or successes you have experienced in working with 
the WBLE. 

Theme One: Learning how to Use the WBLE. Several of the comments related to the 
frustrations associated with the WBLE related to learning how to use it. While many stated that 
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they found the experience worthwhile (as reported in the next section), several reported that the 
learning curve with the tools was (perhaps) more than they anticipated. Some students relayed that 
getting access off-campus was a challenge: 

♦ I am experiencing a few problems accessing the network and sending e- 
mail from home with my Powerbook, but I hope to get it worked out 
this weekend. 

4 I am unable to have Internet access at my home; this means that I need to 
go to a friend's house to access the WBLE. However I am very 
successful operating in the environment once I am in. 

Other students reported that learning how to operate within the WBLE (e.g., finding 
resources) was a challenge: 

Some of my frustrations in the WBLE come from finding my way 
around from place to place. Hopefully, I will become more proficient as 
time goes on. I find when I am working in the system things go fairly 
well, however, when I have been off the computer for a few days, I 
don't always remember what to do 

Theme Two: Invaluable Experience. While many learners reported that working in the 
WBLE "took getting used to," most reported that it was a valuable experience — professionally and 
personally. For some students, working in the ET 533 WBLE meant having to overcome 
personal anxieties associated with electronic communication. However the value in increasing their 
ability to assist their students seemed to outweigh the frustrations and fears. As stated by one of 
the more "technology savvy" students: "I do feel for some of the students who have very limited 
online skills, but then what a wonderful way to learn the needed technology skills so as to pass 
them on to your students." 

Other comments related to the value of the WBLE included: 

♦ I feel it's an invaluable experience. A success for me is feeling like I 
can overcome my anxiety about electronic information sources. 
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♦ Being that I have access from home to the WWW, I have loved it. You 
all have really invested a lot of time setting things up, and I am looking 
forward to taking advantage of this learning opportunity. I'm really 
anxious to see how it all plays out. 

♦ As a player in the future of our district's technology plan, this 
environment interests me very much. One of the most revealing 
discoveries I have made about the WBLE is that it can be both an 
efficient and inefficient means of communication. By this I mean that it 
is inefficient for us to chat about topics by composing our thoughts on 
the computer and sending them to each other.... However, it is also an 
efficient means of communication in that we are forced to write down 
our thoughts. This makes us think them through more clearly and 
precisely state them. I also find it very helpful on the receiving end in 
that I can read and reread something someone else has said. 

Question Three: Discuss “ how it’s going ” overall. 

Theme One: Time Management. Several students reported that they felt considerable time 
pressure in ET 533. Many stated that they felt concern regarding completion of the assignments 
and readings for the course (e.g., "I'm getting out of the blocks kind of slow, and I need the 
weekend to get caught up."). Given that the course was offered during a 6- week condensed 
session in the summer, this was not surprising. However, some of the comments made about time 
management related specifically to the WBLE and interactions within the environment. In addition 
to the quote relayed in the previous section (Question Two, Theme Two), another student had the 
following comment: 

The WBLE is a great idea in theory, especially for dazzling friends and 
family on the wonders of technology, but I haven't really used the chat 
feature enough to feel comfortable with it. When we began working 
with it last Tuesday, I really felt the need to push away form the 
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computer and. give myself to time to digest the questions and the 
information before trying to respond. I felt myself being seduced by the 
power of the moving cursor rather than taking the time to reflect on what 
I wanted to say. 

Theme Two: Going Well. All of the students reported that they thought the course was 
going well overall. Specific examples include: 

♦ It’s going great and I really enjoy the setting. We have a great group 
here. I've found they share the same concerns, frustrations, and 
enthusiasm that I do for our profession. I can't think of a better way to 
grow as an educator. 

4 I am happy to say that it is going better than I expected it to be at this 
point. I was very nervous about taking these courses [library media]. It 
has been a pleasant surprise to be with such enthusiastic and supportive 

people. 

4 I think the class is going great. It is very interesting subject matter, and 
once we become more comfortable with the computer aspect of it ( which 
is happening quicker than I thought it would) it will move along even 

better. 

4 I feel that I have learned many new things. I really enjoy the class, 
especially working on the computers. 

4 I appreciate the opportunity to participate in the process with you 
[Janette], with Landra and Brandon and my classmates. I'm always up 
for new adventures in learning, especially in a technological setting. 

The problems are often similar, but the solutions can certainly be 
revolutionary. Whether this be the case, it has already been very fun 
and advantageous. 
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End of Course Evaluation 

As a part of the end of course evaluation, learners were asked to respond to 4 FOUR 
questions: 

1 . What did you like best about the WBLE experience? 

2. What did you like least about the WBLE experience? 

3. What would you recommend doing differently next time the course is offered as 
a WBLE? 

4. Would you be willing to take another Web-based course? 

The themes evident in their responses presented by question. 

Question One: What did you like best about the WBLE experience? 

Theme One: Convenience and Flexibility. The anytime, anyplace convenience of the 
WBLE was one of the main benefits mentioned by students on the final evaluation. As stated by 
one student: "I loved being able to do my work at home. It was great being able to pick and 
choose the time of day I wanted to work, too." Another student echoed and elaborated this 
statement: 

I loved being able to telecomunicate to class.... I had the opportunity to 
go to some meetings in our district and still attend class! The 
convenience of the environment has a lot to offer. I would not have 
been able to meet all my summer obligations without this environment. 

Flexibility was also closely associated with convenience for many students in the ET 533 
course. As stated by two of the students in the course: 

♦ I certainly appreciated the flexibility afforded by the WBLE. A couple 
of times I was off campus accessing the chat discussion while I was still 
in Longmont getting work done. Having the assignments available 
anytime, anywhere in computer format provide to be a boon on a 
number of occasions. 
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♦ What I liked best about the web based learning environment is its ability 
to be flexible. I enjoyed being able to work on projects at 2:00 a.m. and 
hand then in immediately. 

Theme Two: Potential. Several students mentioned that the potential of the WBLE was a 
positive aspect of the course. Specific examples include: 

♦ I also believe that reference librarians in the future will be 
communicating frequently with their patrons in this sort of environment. 

The practice was greatly appreciated. 

♦ Although we weren't sure what we were getting ourselves into when we 
began this call, I'm REALLY glad to have been given the opportunity to 
be part of this. Once we got use to the format and electronic means, it 
only became second nature, as it suppose to I guess. 

♦ I think there is potential for this type of learning environment, however, 

I think there are things that need to be worked out before this will 
become a standard way of teaching. 

Theme Three: Real Application of Technology. The fact that the course made use of real 
world technologies in real applications was also mentioned as a positive aspect of the course. 

♦ I enjoyed the newness and the fresh approach to the material. This class 
was more than going to the library and looking up material. It was a 
chance to use the technology and learn more about the reference material 
and the WBLE. 

♦ I really liked to see that the new technology is being used. 

♦ I felt that it gave me an opportunity to leant about the material at hand 
( reference) and learn more about the technology used and how to use it 

myself. 
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Question Two: What did you like least about the WBLE experience? 

Theme One: Lack of Confidence in Network. Many students reported feeling uneasy about 
the network — especially in terms of sending and receiving assignments. In general, students 
expressed a concern such as the following: 7 worried about whether or not my assignments were 
received. Other comments included: 

♦ ...I was uneasy about receiving no feedback on the assignments I 
submitted electronically. 1 would have appreciated a reply 
acknowledging that they had been received intact. 

4 I never was absolutely certain that my projects made it "safely" through 
cyberspace. This uncertainty is stressful. 

Theme Two: Technology Challenges. Several students in the ET 533 course also stated 
that technological challenges were a negative aspect of the course for them. The ET 533 WBLE 
was the first experience with WBLEs for all of the students in the course. While some of the 
learners had extensive experience in looking for information via the Web, none had engaged in 
formal learning via the Web. 

Many students related the technology challenges to a lack of experience: 

♦ The only problem I had was that my lack of experience put me at a 
disadvantage. Not only did I have to struggle with the assignments but it 
was an additional task for me to provide them electronically. 

4 We needed more practice on the various ways of communicating. 

Others mentioned specific technologies when discussing technology challenges and the 
aspects of the course that they did not like: 

♦ The group listservfor the last simulation was a bust for me. I never did 
get subscribed even with e-mail help from Brandon. 
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♦ I liked everything except the listserv. I found the mechanics of it 
difficult to use and for me it stifled my thought process. I found myself 
having to stop and start too much. 

For others, the least enjoyable aspect of the course related to unreliable or lack of access: 

♦ The technology is not reliable. Last Thursday the server was down and 
it made me quite nervous. If I had access at another location, it 
wouldn't have been a problem; however, this was the only place I could 
complete the projects. When working in this environment, you 
definitely cannot procrastinate. 

♦ I did not like that I could not access the WBLE from home through most 
of the course. I would have loved to be set up on-line from the 
beginning of the course and would enjoy using this system in the future. 

Question Three: What would you recommend doing differently next time the course is offered as a 
WBLE? 

Theme One: Lots of Feedback. This was the number one suggestion from the students in 
the class: more feedback. For some, a simple reply that the assignment had been received was 
sufficient: "[Set up an] automatic e-mail message indicating that assignments ) have been 
received..." Others requested that the feedback be more detailed, but they wanted to receive 
feedback more frequently and as soon as possible. The students responses to this question directly 
related to their lack of confidence in the network; in fact, often times the two statements were 
linked together (e.g., I don’t feel confident that my assignments are received. A reply message 
back letting me know you received it would be great.). 

Theme Two: More Training on Tools. This question brought responses related to the 
student's feedback on technological challenges: "More training for those of us who have never 
used some of the technologies before ." As stated by one student: 
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♦ If I were taking this class again, I would like to see more explanation on 
how to use all of the mechanics of the system. I was a bit confused at 
the beginning. Possibly a mini course on how to use the site and the e- 
mail would be beneficial. 

Question Four: Would you be willing to take another Web-based course? 

All participants who returned the end of the course survey responded to this question with a 
resounding yes. Specific comments included: 

♦ I would love to take more classes that are Web based. It would be nice 
if we could do more classes off-site since so many of us are traveling 
long distances. 

♦ I really enjoyed the convenience, the practice with the technology and 
the material. 

♦ The advantages of the flexibility far outweigh the disadvantages of the 
technical problems. 

♦ I certainly would be willing to pursue this type of learning again. I 
enjoyed it very much. I felt that it gave me an opportunity to learn about 
the material at hand ( reference ) and learn more about the technology 
used and how to use it myself I found this type of learning 
atmosphere a pleasure to participant in. 

Implications and Recommendations 

Working in WBLEs is an exciting area — from several perspectives: designer/developer, 
facilitator, and researcher. While we have made considerable strides in the last eight years since 
the first Web pages were released in Switzerland, we still have many challenges to face and hurdles 
to overcome. 

One of the responsibilities for those of us involved in distance learning, regardless of the 
delivery system used, is to prime the expectations of student participants. In the ET 533 course. 
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we stressed that WBLEs are not intended to replicate the traditional face-2-face learning 
environment. We also talked a great deal about the “experimental” nature of the course, and how 
student feedback would indeed provide a means for improvement and revision, both during the 
course, and afterward. This emphasis appeared to predispose students to consider the “oopses” 
we encountered as learning experiences rather than obstacles hindering their progress. 

It also is important to note that the instructors knew that they had to be flexible in terms of 
expectations regarding timelines, students performance, and content when the technology thwarted 
“the plan.” The modeling was crucial to students’ willingness to participate and take the risks 
associated with a new learning environment. As stated by Mr. Murry, “model the process and give 
one-on-one assistance” when seeking solutions to individual learner challenges in WBLEs. 

Practice and a friendly reminder can also go a long way toward increasing learner comfort and 
willingness to continue to participate. 

Another implication for WBLEs relates to course size in terms of participants. We had 
twelve (12) students in the ET 533 course. This appeared to be a comfortable number given the 
level of interaction in the course and the course timeframe (six (6) weeks). However, several 
questions remain to be answered: (1) How many more students could we have enrolled and still 
provided effective support and promoted learning? (2) Should there a cap imposed on WBLEs? 
and, a question closely related to the number of participants, (3) Should more than one facilitator 
always be involved in WBLEs to ensure effective support and engagement with the learners? The 
answers to these questions seem imperative if we are to continue to develop meaningful learning 
environments via the Web. 

A final implication relates to hardware/software challenges encountered in the ET 533 
WBLE. We experienced several challenges in designing and implementing the course Web site. 
However, we do not feel we are unique in our challenges (see Khan (1997) for other case studies 
related to WBI). It is imperative that we continue investigation of hardware/software issues related 
to WBLEs if we are to ensure the long-term viability of this medium for the distance delivery of 
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Figure 1. ET 533 Home Page. 
Figure 2. ET 533 timeline. 

Figure 3. ET 533 discussion area. 
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